DRAFT Spec for Buffer Exporting and Logging #11090

zadjii-msft · 2021-08-31T16:04:40Z

⇒ doc link ⇐

Summary of the Pull Request

This is an intentionally brief spec to address the full scope of #642. The
intention of this spec is to quickly build consensus around all the features we
want for logging, and prepare an implementation plan.

Abstract

A common user need is the ability to export the history of a terminal session to
a file, for later inspection or validation. This is something that could be
triggered manually. Many terminal emulators provide the ability to automatically
log the output of a session to a file, so the history is always captured. This
spec will address improvements to the Windows Terminal to enable these kinds of
exporting and logging scenarios.

PR Checklist

Detailed Description of the Pull Request / Additional comments

*^*_* read the spec _*^**

Open Discussion

What formatting string syntax and variables do we want to use?
How exactly do we want to handle "log printable output"? Do we include backspaces? Do we only log on newlines?
> maybe consider even simpler options like just ${date} and ${time}, and allow for future variants with something like ${date:yyyy-mm-dd} or ${time:hhmm}

carlos-zamora · 2021-08-31T17:47:44Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+  on close? on newline? It's unclear exactly when PuTTY flushes with this off.
+  Need more coffee. -->
+
+### Future Considerations


Ideas/Considerations:

Text formatting

Does the output file record the formatting of the text (i.e. color, font, etc.)? Or is the output ".log" file just a txt file?

If we already can output formatted text, we should add this as a possible configuration (later imo, just do plain text for now).

Maybe we could output to an ".html" file using TextBuffer::GenHTML?

Super logging (no idea what to call this, but it needed a cool name)

I remember @DHowett created a cool thing one time where you could record and play back a session in Conhost. I don't remember the specifics. But that seems like something that could be a part of this logging discussion.

Nope, it's just text.

Cool idea, I'll throw it on Future Considerations

I think you're thinking of asciicast/asciinema which we're tracking in Feature request: Recording #469. I'll link the two up.

Dustin's thing recorded raw API calls from the driver. It was awesome, but absolutely overkill for this feature 😄

carlos-zamora · 2021-08-31T17:49:46Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+
+I'm proposing the following actions and profile settings
+
+* New Action: `exportBuffer()`.


Suggested change

* New Action: `exportBuffer()`.

* New Action: `exportBuffer`.

carlos-zamora · 2021-08-31T17:54:04Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+  - `path`: Same as the `path` in the `ExportBufferArgs` above
+  - `append`: Same as the `append` in the `ExportBufferArgs` above


Suggested change

- `path`: Same as the `path` in the `ExportBufferArgs` above

- `append`: Same as the `append` in the `ExportBufferArgs` above

- `path`: Same as the `path` in the `exportBuffer` above

- `append`: Same as the `append` in the `exportBuffer` above

ExportBufferArgs wasn't introduced... but I guess you also mean to create a mechanism like NewTerminalArgs

Oh yea derp, I just kinda implied that ExportBufferArgs were the ActionArgs for the exportBuffer action.

carlos-zamora · 2021-08-31T19:24:55Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+  - `captureAllOutput`: (boolean, defaults to `false`) When true, don't log only
+    printable characters, also log non-printable escape characters written to
+    the Terminal.
+  - `captureInput`: (boolean, defaults to `false`) Additionally log input to the
+    Terminal to the file. Input will be formatted as the traditional VT
+    sequences, rather than the full `win32-input` encoding.


Idea:

remove captureAllOutput and captureInput

(names are still a work in progress, but you get the idea) add captureHiddenData: (flag enum { none, vtKeyboardInput, vtMouseInput, vtOutput, all })

carlos-zamora · 2021-08-31T19:25:25Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+  - `newFileEveryDay`: (boolean, defaults to `false`) This requires the `day` to
+    be an element of the path format string. When logging with this setting,
+    opens a new file at midnight and starts writing that one.
+  <!-- TODO! - `flushFrequently`: (boolean, defaults to `true`) -->


Curious, why?

Meh, I found it in the PuTTY docs, thought it was interesting, but commented out because I hadn't investigated it, then forgot to remove the comments before shipping the PR. Will move to future considerations, since it doesn't seem critical to the design.

carlos-zamora · 2021-08-31T19:54:50Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+My worry with logging the backspaces is that conpty is sometimes a bit noisier
+than it needs to be with using `^H` as a cursor positioning sequence. Should we
+only log lines when the cursor newlines or otherwise moves from the line it is
+currently on?


Yeah I think we should only log when we move to another line. Then just get whatever text was at that line and throw it in a file.

What happens in vim then? Or when performing a cls? I guess the mechanics of this we'll have to compare with PuTTY a bit closer.

Not sure if I understood the discussion correctly, but we must be able to log everything that happened. Even within the same line, and even if it is noisy. These logs are sometimes required for investigating/debugguing.

Since there might be too much updates we will need to use some async writes with buffering (and ensure everything is flushed when closing).

Probably it should be configured by verbosity level

I guess my real concern is "does backspace count as a printable character". Yes-ish? By itself it's a non-destructive character. But if we don't treat it as printable, then asdfBkspBksp gets emitted as asdf^h ^h^h ^h, but then would get logged as asdf (with two spaces at the end there). So I guess, backspace should count as printable.

The "log everything" option is frankly just way easier to implement than the "printable characters only" one. Maybe we start with that one 😛

carlos-zamora · 2021-08-31T19:56:56Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+should we log the current buffer contents as well?
+
+I'm inclined to lean towards simply "all future output", and ignore any current
+buffer content. If the user rally wants to log the current buffer contents _and_


Suggested change

buffer content. If the user rally wants to log the current buffer contents _and_

buffer content. If the user really wants to log the current buffer contents _and_

carlos-zamora · 2021-08-31T19:58:08Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+started logging with `toggleLogging`, what should we log? All future output? Or
+should we log the current buffer contents as well?
+
+I'm inclined to lean towards simply "all future output", and ignore any current


100% agree. In my mind, toggleLogging is intended to be "I want to start writing to the file now. Up until this point, I was setting up the terminal to be in a good state".

carlos-zamora · 2021-08-31T19:59:53Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+`exportBuffer` to a file, then `toggleLogging` to that same file with
+`"append":true`.
+
+## Potential Issues


Ideas/Considerations:

How will logging work if you resize during a logging session?

carlos-zamora · 2021-08-31T20:11:52Z

doc/specs/#642 - Buffer Exporting and Logging/#642 - Buffer Exporting and Logging.md

+<!-- * [ ] `LoggingSettings` property for "Flush log frequently", defaults to
+  `true`(?). This causes us to flush all output to the file, instead of just...
+  on close? on newline? It's unclear exactly when PuTTY flushes with this off.
+  Need more coffee. -->


Did you get more ☕? haha

WSLUser · 2021-09-02T13:04:03Z

I hope HyperTerminal from XP is also considered as example. I would like to see the same feature is available (for example you can choose when to start and stop the session and the log file is generated in plain text so you can read it in Notepad).

zadjii-msft · 2021-09-02T13:11:39Z

you can choose when to start and stop the session and the log file is generated in plain text so you can read it in Notepad

Yep, that's already in the spec.

This adds an action for the context menu entry we added in #11062. That PR added support for exporting the buffer, exclusively through the tab item's context menu. This adds an action that can additionally be bound, which also can export the buffer to a file. This action accepts a `path` param. If empty/ommitted, then the Terminal will prompt for the file to export the buffer to. * Does a part of #9700 * Spec in #11090, but I doubt this is contentious * [x] This will satisfy #12052 * [x] I work here * [x] docs added: MicrosoftDocs/terminal#479

…R backlog. We're not making any progress on this immediately.

zadjii-msft · 2022-12-12T22:52:32Z

I'm officially moving this to the drafts/ folder to clear out the PR backlog. We're not making any progress on this immediately.

carlos-zamora

DRAFT!!

ghost · 2022-12-13T16:32:45Z

Hello @zadjii-msft!

Because this pull request has the AutoMerge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.

zadjii-msft and others added 2 commits August 31, 2021 10:58

Write the entire exporting & logging spec

b0006f4

Migrate spelling-0.0.21 changes from main

85eead7

This comment has been minimized.

Sign in to view

zadjii-msft added Issue-Docs It's a documentation issue that really should be on MicrosoftDocs/Console-Docs Product-Terminal The new Windows Terminal. labels Aug 31, 2021

zadjii-msft mentioned this pull request Aug 31, 2021

megathread: Buffer exporting and logging #9700

Open

12 tasks

carlos-zamora reviewed Aug 31, 2021

View reviewed changes

fischerdouglas mentioned this pull request Sep 1, 2021

Feature request: export console history as txt #642

Closed

zadjii-msft self-assigned this Sep 13, 2021

This was referenced Jan 5, 2022

[Terminal > Export Text]: Keyboard only users do not have option to use 'Export text' functionality. #12052

Closed

Add an exportBuffer action #12097

Merged

zadjii-msft added 2 commits December 12, 2022 16:51

Merge remote-tracking branch 'origin/main' into dev/migrie/s/642-logging

a317d53

I'm officially moving this to the drafts/ folder to clear out the P…

0dd9cbd

…R backlog. We're not making any progress on this immediately.

zadjii-msft removed their assignment Dec 12, 2022

zadjii-msft changed the title ~~Spec for Buffer Exporting and Logging~~ DRAFT Spec for Buffer Exporting and Logging Dec 12, 2022

This comment has been minimized.

Sign in to view

many typos

457e3c4

This comment has been minimized.

Sign in to view

carlos-zamora approved these changes Dec 12, 2022

View reviewed changes

DHowett approved these changes Dec 12, 2022

View reviewed changes

the bane of my existence

c9ec266

zadjii-msft added the AutoMerge Marked for automatic merge by the bot when requirements are met label Dec 13, 2022

ghost merged commit 274bbf0 into main Dec 13, 2022

ghost deleted the dev/migrie/s/642-logging branch December 13, 2022 17:06

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DRAFT Spec for Buffer Exporting and Logging #11090

DRAFT Spec for Buffer Exporting and Logging #11090

zadjii-msft commented Aug 31, 2021 •

edited

Loading

This comment has been minimized.

carlos-zamora Aug 31, 2021

zadjii-msft Aug 31, 2021

carlos-zamora Aug 31, 2021

carlos-zamora Aug 31, 2021

zadjii-msft Aug 31, 2021

carlos-zamora Aug 31, 2021

carlos-zamora Aug 31, 2021

zadjii-msft Aug 31, 2021

carlos-zamora Aug 31, 2021

zadjii-msft Aug 31, 2021

Don-Vito Sep 1, 2021 •

edited

Loading

zadjii-msft Sep 1, 2021

carlos-zamora Aug 31, 2021

carlos-zamora Aug 31, 2021

carlos-zamora Aug 31, 2021

carlos-zamora Aug 31, 2021

WSLUser commented Sep 2, 2021

zadjii-msft commented Sep 2, 2021

zadjii-msft commented Dec 12, 2022

This comment has been minimized.

This comment has been minimized.

carlos-zamora left a comment

ghost commented Dec 13, 2022


		I'm proposing the following actions and profile settings

		* New Action: `exportBuffer()`.

	* New Action: `exportBuffer()`.
	* New Action: `exportBuffer`.

		- `path`: Same as the `path` in the `ExportBufferArgs` above
		- `append`: Same as the `append` in the `ExportBufferArgs` above

	buffer content. If the user rally wants to log the current buffer contents _and_
	buffer content. If the user really wants to log the current buffer contents _and_

DRAFT Spec for Buffer Exporting and Logging #11090

DRAFT Spec for Buffer Exporting and Logging #11090

Conversation

zadjii-msft commented Aug 31, 2021 • edited Loading

⇒ doc link ⇐

Summary of the Pull Request

Abstract

PR Checklist

Detailed Description of the Pull Request / Additional comments

Open Discussion

This comment has been minimized.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Don-Vito Sep 1, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WSLUser commented Sep 2, 2021

zadjii-msft commented Sep 2, 2021

zadjii-msft commented Dec 12, 2022

This comment has been minimized.

This comment has been minimized.

carlos-zamora left a comment

Choose a reason for hiding this comment

ghost commented Dec 13, 2022

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

zadjii-msft commented Aug 31, 2021 •

edited

Loading

Don-Vito Sep 1, 2021 •

edited

Loading

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.